On the asymptotic rate of non-ergodic information sources

Before proceeding to the formulation of the problem we are treating in this paper, we shall remind some simple facts upon which the concepts given in the sequel are based. Let us imagine such a situation when a receiver is expecting one of the messages z belonging to a finite set Z of messages which are possible to come under the given situation. The uncertainty of the situation described is evidently the greater, the larger is the number of all a priori possible messages; let us denote the latter number, i.e. the number of messages in Z, by \z\. Originally the quantity of information which is needed to remove this uncertainty, is numerically expressed by the number log \z\, where the logarithm.is taken to the base 2; in other words, log \z\ represents the quantity of information expressed in bits which is contained in any message z in Z reaching the receiver. Consequently, the number log \Z\ as a measure of uncertainty may be called the logarithmic uncertainty of the set Z.